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ABSTRACT 



Four types of science education research are 
defined: empirical, philosophical, policy, and developmental studies; 
problems and approaches related to each type are analyzed. Problem 
definition is discussed in terms of theoretical and practical 
problems. Arguments are presented for and against requiring doctoral 
students to define their own problems. Use of levels of significance 
in empirical research is discussed and a distinction drawn between 
statistical significance and decision making significance. The need 
for philosophical studies which analyze assumptions and the 
consequences of actions is stressed. Historical and comparative 
studies are advocated as aids to making policy decisions. A pattern 
for developmental research is outlined. The responsibility of the 
researcher to suggest the implications of all types of research is 
discussed. (EB) 
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What is science education research? What are the approaches 
that we take to research in science education? How can our research 
be improved? Periodically, it is useful to reflect upon our ex- 
periences in our field, and since much of our research is still carried 
OUj by doctoral students, it is also important to reflect upon our 
experiences in guiding doctoral students in their research experiences. 1 



Science education research is the systematic attempt to de- 
fine and investigate problems involved in learning and instruction 
in science Pour different types of science education research can 
^r> ine Z 1 empirical, ( 2 ) philosophical, (3) policy and, ( 4 ) 

° r forrnative Sadies. 2 These types are interrelated 
and more than one approach to research is often used in a study. 

r example, developmental studies often involve empirical investi- 
gations, but the emphasis is on the development of instruments or 
instructional procedures and materials; similarly, empirical studies 
often involve the development of instruments or materials, but the 
emphasis is on the empirical investigation. 



Each of these types of research will be defined, and some of 
e critical problems that we are encountering in each of these 
jypes of research will be briefly analyzed. First, however, it is 

important to consider "problem definition" for this is central to 
all research. 



. . . Problem definition is probably the most 
critical, difficult and frustrating dimension of research. Pro- 
blems are defined out of problematic situations. We may sense 
that a situation is not as it might be, "feel" that there is 
something wrong, suspect an inconsistency, or wonder why some- 
ing happens. But, to do effective research, the problem 
must be seen more clearly and defined more precisely. 



. *P & P er presented at the Annual Luncheon Meeting of the 
National. Association for Research in Science Teaching, March 
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Eventually, the problem can often be stated in the form of a simple, 
clearly understood question. An inchoate and messy problematic 
situation seems much clearer after the distractions and irrelevancies 
have been peeled away. Once a problem has been clearly defined, the. 
procedures are often a matter of logical deduction. Problem definition 
may be the most difficult and creative dimension of research. But, 
to state a problem clearly and succinctly is to be well along in in- 
quiry. 



There are two major sources of problems in science education 
that may be called the theoratical and the practical . In the theo- 
ratical , some lack, inconsistency, or vagueness is sensed as theory 
is applied in science education. For example, for several decades 
investigators have been studying conservation behavior among child- 
ren using tasks drawn largely from mathematics and the physical 
sciences. We have recently recognized our lack of understanding 
of how children "conserve” using materials drawn from the biological 
sciences and ha,ve set out to develop and test such life science 
tasks. In the practical, some problem is sensed as we work with 
students, or struggle with program planning, or feel a lack of 
effective science materials. For example, there are many science 
educators who sense the practical need for developing more eff- 
ective ways of stimulating the continuing intellectual, growth of 
teachers. It is essential that we continue the struggle to de- 
fine problems from both theoretical ana practical problematic 
situations. 

In both the sciences and education we are learning that it 
is often important to take a philosophical view in order to identify 
significant problems. In the sciences, the philosophical notions 
of symmetry and simplicity have helped stimulate the sense that 
something is not as it might be and to define the problem that can 
be tackled. In education and politics the "grass roots model" has 
been a useful approach. In this approach to problem definition, 
the possible effects of a proposed policy or practice upon an in- 
dividual child, teacher or school system are deduced. Similarly, 
we can sometimes see problems more clearly if we shift reference 
frames and try to view problematic situations from different per- 
spectives. The education of disadvantaged children, for example, 
is viewed differently by the struggling, harrassed mother who 
still generates hope for her children and the scholar who views 
the problematic situation in terms of the generalizations of his 
discipline. Group discussions are another way that the nature of 
the problems can be clarified as different individuals approach a 
problematic situation from different reference frames and help the 
individual to see his problem more clearly. Sometimes, a philo- 
sophical consideration of the ideal can help in problem delineation 
as someone has said, "Once we know what we want, we can find ways 
to get it". A philosophical view of a problematic situation can 
sometimes lift our efforts above mechanical puzzle solving to deal 
with that which is more significant. 
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To what extent should doctoral students who are preparing to 
Become researchers become immersed in the difficult, time-consuming, 
and frustrating process of problem definition? This is a matter of 
great concern to those with responsibility for the development of 
doctoral programs, and there are at least two sides to the issue: 



Pro: 1. If problem definition is the most diff- 
icult and important dimension of research, 
then the future researcher in science 
education should have experience in the 
struggles , frustrations and satisfactions 
of problem definition. 

2 . The philosophical analysis of a problem- 
atic situation in order to define the pro- 
blems is an extremely important educational 
experience in which a student gains a more 
profound understanding of his field. Ex- 
perience in problem definition may help 
students to become creative researchers 
rather than technicians. 

3. It is important that a doctoral student be 
identified with his problem and believe in 
its importance . To spend two or more years 
working on a problem that is not your own 
and to live with it for the rest of your^ 
life can be a painful and limiting experience. 



4 . Although senior science educators have ha^ 
more experience , the field will benefit 
from the fresh , brash questions that are 
often asked by the relatively unsophisticated. 

Con: 1. Problem definition requires long and 

intensive experience in the field and, 
perhaps, can be done more effectively 
and efficiently by the doctoral adviser. 

2. Many of our most significant problems are 
too big to be tackled by a single investi- 
gator and if we are to make progress in 
science education we must have teams of 
investigators tackling these larger prob- 
lems. The field has benefited very little 
from many individual, unrelated research 
efforts and needs large scale team efforts 
under the leadership senior researchers, 

3 . The primary function of a doctoral program 

is to develop research techniques and skills, 
and this can best be done on a managable pro- 
blem that may be a part of a team effort. 



4 . 




The interests and competencies of the sponsor 
of the doctoral study are factors to be con- 
sidered in problem selection. Most sponsors 
are not competent to guide all kinds of edu- 
cational research, and a student will gain 
more from working in areas in which his 
sponsor is able and interested. 



It is almost a truism in education that it is insufficient to 
define problematic situations into "either-or” propositions. We 
will undoubtedly continue to have some doctoral students struggle 
with problem definition and hand other students problems that are 
parts of larger undertakings. And it maybe that, if we continue 
to recognize individual differences , we should use different 
approaches with students of differing aptitudes and proclivities. 
However, if we continue to be concerned with the education of 
creative researchers and leaders in science education, we should 
probaoly increase the number of students who view science edu- 
cation philosophically and struggle to define the problem out 
the inchoate problematic situation. Once having had this ex- 
perience, ishey may be more likely to continue to do research 
throughout their careers. 

Empirical research. Empirical studies usually involve the 
collection of data concerning the behavior of students, teachers 
or other subjects under study. Often, the data are used to accept 
or reject hypotheses. We are encountering a number of problems 
with empirical studies, and there are opportunities that are not 
being aggressively pursued. 

Often, problems are defined in such a way that great efforts 
lead to seemingly meaningless or relatively inconsequential results. 
The classic example involves the large number of studies of the 
relative effectiveness of a laboratory approach as contrasted to 
a lecture-demonstration approach to science teaching. There have 
been over one hundred studies of this question, and the results 
have been inconclusive. Obviously, a good science teacher will 
use both laboratory and lecture-demonstration approaches to teach- 
ing, as well as discussions, readings, field trips, audio-visual 
materials-, lectures, individual counseling and other approaches. 

In research, it would be more useful to study what approaches 
are most effective under what conditions to achieve specific 
objectives with various kinds of students. However, if the 
tremendous energies that have gone Into the laboratory vs. 
lecture- demonstration studies have helped us to define our pro- 
blems more clearly, they have not been in vain. On the other 
hand, if more attention had been given to philosophical analysis 
and problem definition, it is conceivable that we might have 
moved ahead faster and more efficiently. 



o 
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A disturbingly large number of science education studies have 
reports of no significant differences. Most investigators expect 
to find significant differences, otherwise there would be little 
reason to undertake the studies, and the researcher dutifully re- 
ports his findings but has difficulty in masking his disappointment. 
Usually, few take cognizance of findings of "no significant differ- 
ences" because the wrong problem may have been posed, or because we 
simply cannot afford to stop using new (or old) approaches to teach- 
ing and educational materials, or because we believe that the diff- 
erences are significant enough for us. 

In many cases the finding of "no significant differences" is 
rooted in the nature of man and his experiences. The individual is 
a product of his inmate potentialities and all of his past experiences. 
To design an experiment in which one variable among many influential 
factors is manipulated over a period of a year or less, to collect 
data using one of the few instruments available, and to expect signif- 
icant differences to occur is unrealistic and possibly undesirable. 

The effect of the one variable manipulated over 75 hours or less in 
a classroom setting where there may be many more potent influences 
at work iB miniscule when contrasted to the bank of experiences 
that even young children have had in the past. If students were 
to make the marked changes in behavior or attitudes that we 
hypothesize as a result of the limited educational imput in some 
of our studies might even be considered a sign of instability. 

However, when students begin study in areas where they have had 
limited previous experience such as in the study of a foreign 
language or chemistry we usually find significant changes in 
scores or measures of achievement closely related to these areas. 

We should not abrogate our responsibility to make judgements 
in the choice of levels of significance. Too often, levels of 
significance a w e set mechanically at the . 05 level without recog- 
nizing that levels that we wish to accept will vary and that in 
education there are risks involved in rejecting as well as accept- 
ing hypotheses. Clearly, the level of significance that we will 
accept in the testing of a method of teaching or new educational 
materials will be different from that used in testing drugs or 
airplane parts. However, we have often accepted uncritically a 
level of significance that may be appropriate in another field 
but conceivably unwarranted in our educational research. The 
level of significance indicates the probability that the observed 
result could have been produced by chance rather than by our 
treatment. In educational research we might often be justified 
to take the risk that the favorable results we observe in using 
a new science program or teaching procedure might have been due 
to chance rather than the program or procedure. In many cases 
in education, the consequences would not be catastrophic. 

We should also recognize that there is risk in rejecting new 
approaches and materials. To reject new approaches or materials 
because changes as measured by admittedly imprecise instruments do 
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not meet mechanically set levels of significance can certainly 
demoralize creative innovators and possibly deprive students of 
educational experiences that could be of value. 

In an applied field such as science education ¥e might con- 
sider using both statistical significance and decision-making 
significance. 3 Certainly, we will not reject the use of "open- 
ended” approaches to laboratory instruction or field trips in 
earth science teaching because the difference in achievements 
of students when these approaches are used are not found to be 
significant, at the .05 level. Perhaps, we should set levels 
of significance for dec is ion -making. It might be possible have 
knowledgeable practitioners help set the level at which they 
would accept or reject an hypothesis. Like statistical signif- 
icance, the level of decision-making significance might be set 
before the investigation is undertray. The development of levels 
of decision-making significance might lead practitioners to pay 
more attention to research and give our research efforts greater 
impact. 



Research should be a creative undertaking and sometimes the 
unexpected findings are of the greatest significance. In a classic 
example, Oersted's discovery that a compass needle was affected by 
an electric current in a nearby conductor was of far greater sig- 
nificance than his more prosaic activities on that historic day. 

In science education, Lampkin’s unexpected discovery of variability 
in the recognition of scientific inquiry was a much more important 
finding than the discovery of any patterns in thg treatment of 
scientific inquiry in textbooks would have been. There is a re- 
grettable tendency in educational research to develop a tight 
research design, usually conservative and involving little risk, 
and then to patiently grind out the research, reporting findings 
that surprise no one. Certainly, there are many, often unpre- 
dictable, factors that affect almost any educational endeavor, 
and education is such a "soft" science that we can almost expect 
something unusual to happen as we carry out educational research. 

The creative researcher is alert to the unexpected, and our 
literature will be enriched as the unforeseen is reported. 

If fields of research pass through stages of evolution, 
there would be general agreement that education is at an early 
stage of development. One of the early stages in most fields 
of science is a natural history stage in which a great deal of 
effort is expended on the description of phenomena. These de- 
scriptive studies make possible the development of classificatory 
systems and eventually postulations which can serve as a basis for 
hypothetico-deductive studies, Do we have enough descriptions of 
how children learn? Of how teachers teach? In many cases we may 
not even know what variables should be considered, and these might 
be uncovered through natural history studies. 

To carry out natural history studies' will necessitate the greater use 
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of anecdotal records, observational protocols, case study techniques 
and, ae in anthroprology , call for long, book-length research reports, 
it will also mean that in our research reviews we should use scanning 
procedures that will recognize these kinds of reports as well as 
journal articles. 

Philosophical studies . Philosophical studies may involve the 
analysis of assumptions underlying positions or actions, the nature 
ot problematic situations and the delineation of problems within 
problematic situations, and of possible consequences of proposed 
actions. To a certain extent, philosophical studies may be seen 
as the application of intelligence to our problems. The most potent 
criticism to be made of this type of science education research is 
that there has not beexi enough of it. 

Certainly, we need rigorous analysis of the assumptions that 
underly our present science programs. At times it would appear 
that program planners hold that the science subjects in the high 
school curriculum and their boundaries were handed down from on 
high rather than having been decided upon by fallible human beings 
in lraphazard ways. For example, there must be some inconsistency 
in the fact that in a nation that has taken epoch-making steps 
in the exploration of the universe where science educators solemnly 
declare that the study of science should help young people develop 
a world view and few students will have an opportunity to make a 
systematic study of astronomy. This is one of many issues related 
o science programs that deserves rigorous examination. 

One of the functions of philosophical analysis is to clarify 
possible consequences of actions. Certainly, it is a characteristic of 
intelligence to try to predict possible consequences. In fact, some 
pschologists might say that this is a characteristic of operations at 
the formal level of intellectual development. Some philosophers have 
suggested that the analysis of possible consequences in social situ- 
ations leads to postulations from which we can deduct hypothetical con- 
sequences and the tests to determine the extent to which these hypo- 
thetical consequences actually do occur. In other words, the analysis 
of possible consequences lays the foundations for social sciences, and 
most research in science education can be considered social science 
research. While the mechanistic, reductionist philosophies of science 
have been of great utility in the physical sciences, in dealing with 
social situations where complicated human beings both as individuals 
and as groups occupy the center stage and where the whoXe is almost 
always greater than the sum of the parts,, the philosophy of pragma - 
txcism with its emphasis upon the analysis of possible consequences 
provides r promising base. 

, philosophical analysis of possible consequences might serve to 

dampen the extrema swings of the pendulum in our field. For example, 
the severe criticism of science teachers and educators as lacking in 



rigor and possessing low standards led some of our more pliable colleagues 
to introduce kinds of rigor and standards that even the most savage critic 
could not have wanted. Part of the retreat from society, rebellion against 
our culture and flight from, science on the part of some of our most xntelli 
gent and sensitive young people can be attributed, at least in part, to 
this highly unfortunate extreme swing of the pendulum. At the present time 
we may be seeing young people who are almost crying out for a form of 
discipline — but a discipline rooted in our experience in analyzing situ- 
ations and suggesting possible consequences. For example, in dealing, 
with the environment there is the clear necessity to analyze the possible 
consequences of our actions, and the sudden surge of concern for the 
environment on the part of young people calls for leadership in the 
analysis of situations for the possible consequences of proposed actions. 
The philosophically inclined science educator has an opportunity of. 
abstracting from such experiences a discipline for analysis and action. 

There is a need to place our research and its results in perspective, 
to relate it to other kinds of experiences , and to extend the results to 
logical implications for action. For example, many researchers apparently 
do not believe their research because their actions are not consistent 
with their findings. But, there is always a danger that people outside 
the field of education will take the results much more seriously. Some 
educators have "discovered” that education isn't as potent a force. for 
individual or societal development as some enthusiasts have maintained. 
These "findings” have been used by critics to downgrade the importance 
of education. However, some of these educators will move heaven and 
earth to make certain that their children get a good education. What 
interpretation should be placed on these findings? On the other. hand, 
the implications of findings are sometimes not extended into action. 

For example, an analysis of developments in agricultural technology 
indicated that there could be a mass exodus from rural areas to the 
cities. A logical extension of these findings to action could have 
prevented some of the tragedies that are now unfolding. Philosophical 
analysis could add meaning to our research and make more likely the 
responsible utilization of its results. 

Policy studies . Policy studies in science education include 
historical studies in which policy decisions and their consequences 
are analyzed, comparative science education studies in which. we try 
to learn from the experiences of others, and futuristic studies. in which 
we try to project possibilities and predict consequences of various 
proposed policies for the future. The years since 1955 have seen 
more activity in science education than any other comparable period; 
it would be unfortunate if we were not to make every effort to pro- 
fit from this experience. Hopefully, we will have new policies and 
grander efforts in the future; certainly, we should use as much in- 
telligence as we can muster to insure that we will do better in the 
future than we have in the past. 
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Historical studies should be viewed primarily as systematic attempts 
to learn from the experiences of the past. Too often,, historical studies 
have served a conservative function to show all the things that have been 
tried in the past with limited success forgetting that usually no new 
venture is an exact duplicate of a past experience. A much more pro- 
fitable approach to historical studies is to seek for ideas and to fry 
to learn from what has happened in the past. Undoubtedly, we would 
have been more efficient and effective in the past decade if more 
systematic attempts had been made to profit from the experiences of 
insightful experimenters and clever innovators of the last few decades. 

Usually, the most valuable historical studies are critical and 
analytical as well as objective* Sometimes, historical studies are 
written by those who played a central role in an educational develop- 
ment and read like an apologia for what was done. While these ’’memoirs” 
can be valuable source materials for the historian, they are no sub- 
stitute for analytical history. 

Through comparative science education we can learn from the ex- 
periences of others. To give one example, India is embarked upon an 
unparalleled educational effort to check population growth. Other 
nations should do the same, and we have much to learn from the plan- 
ning and programs, successes and failures of our Indian colleagues. 

More science educators should be encouraged to undertake comparative 
science education studies when they visit and study in other nations, 
and we should develop better mechanisms for reporting this research. 

There is a rich lode of experience in science education in other 
nations, and we can learn a great deal from an analysis of these 
efforts 

More attention needs to be given to futuristic studies in science 
education. In other fields, imaginative scenarios are developed that 
explore the possibilities and consequences of various proposed policies. 
The development of such scenarios sometimes make explicit possibilities 
that would otherwise not be foreseen. It also makes possible more in- 
telligent choice between alternatives. There have been very few such 
futuristic studies in science education. Greater effort in this direction 
might help us to develop more effective policies in the future. 

Developmental research. Developmental studies involve the 
preparation of new educational materials, procedures, or programs 
and systematic tryouts in which feedback is gathered that can be 
used for improvement. They have the attraction that they can lead to 
a perceptible improvement in the education of students. At a time 
when there is a call for ’’relevant” research and education this is 
an important consideration. Although developmental research has not 
had as prominent a role in the course content improvement projects as 
we might wish, some valuable insights have been gained. 
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A general pattern for developmental studies has evolved. The 
pattern that follows is often altered to meet special conditions, 
but the major elements are usually included: 

1. Exploration for ideas. The innovators usually have a basic 
idea that they want to develop and test. However, it is 
useful to search the literature for additional, ideas that 
may make the development more useful and to interview 
others who have a record of innovative work or practical 
experience in the field. 

2. Preliminary planning. The innovators outline the content 
of the innovation, assign responsibilities for development, 
and plan procedures for development and testing. 

3. Development of preliminary form of innovation. The developers 
write or build the innovation. Drafts or preliminary models are 
often discussed and criticized in in-house seminars. Where app- 
ropriate, laboratory tests are run to make certain that the in- 
novation "works”. The content validity is checked. 

k. Small scale tryouts. The preliminary form of the innovation is 
tried by a small group of teachers under field conditions in 
which the innovators are deeply involved. It is helpful if the 
teachers and others involved in these tryouts have some under- 
standing of the total enterprise and no reluctance to express 
their suggestions and criticisms. 

5. Preparation of the trial form of the innovation. Engineering and 
production problems involved in large scale manufacturing may be 
considered in the preparation of the trial form, 

6. Field trial of innovation. The innovation- is.' tried with. populations 
that are believed to be roughly representative of the total pop- 
ulation that may use the innovation,. The general appropriateness 
and applicability of the innovation are reviewed. Some new ideas 
may be generated out the experience in a variety of localities. 

In some cases, summative research is carried out to establish 
achievement norms for different populations. Problems of distrib- 
ution and of use under general field conditions are detected. 

7. Preparation of version for dissemination. Revisions are made in 
light of field trials, and a version for general dissemination 
is prepared. Often, this version is a commercial edition. 

8. Analysis of experience. Those engaged in developmental studies 
have an obligation ho analyze their experiences so that others may 
learn from them. 
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In terms of process, our experience seems to indicate that the 
interchange in the small scale tryouts between innovators and with 
teachers or students who have some understanding of what is being 
attempted is especially useful. It is very difficult for someone 
who has not had an opportunity to become aware of the total nature 
of the innovation and the purposes it is to serve to make many help- 
ful suggestions or criticisms on a feedback form. .Articulate coop- 
erating field testers who have an understanding of the innovation 
can make many helpful criticisms and suggestions in group discussions. 



There is no mention of evaluational research in which the effects 
of using an innovation and other materials or procedures are com- 
pared. Sooner or later, teachers and instructional leaders have 
to make decisions as to the choice of materials and procedures, 
and it would be desirable to have these decisions based upon re- 
search. It is difficult, and usually suspect, for the developers 
to undertake this kind of evaluational research. An argument can 
be made for having independent groups undertake this kind of evalu- 
ation. On the other hand, decisions about the use of innovations 
almost always have to be made in light of local conditions. In 
many cases, it may be more desirable to have school systems and 
other consumers evaluate innovations by undertaking systematic 
pilot tests of innovations under local conditions. 



While we 210 doubt should continue our efforts to develop better 
science programs, there are important reasons for suggesting that 
additional effort be devoted to the development of units that can 
be used to build programs. The high mobility of the American pop- 
ulation which results in some schools having an annual turnover of 
students exceeding 100$, the high absentee rate in many of our urban 
schools, the difficulty that students encounter if they have missed 
some key element of a tightly and logically organized program, and 
the desire of local communities to develop science programs that 
they believe best meet their needs are arguments for the develop- 
ment of smaller units of instruction. 



We need developmental studies that will help us in our research. 

A wider range of effective research instruments and tools is needed. 
Science education researchers have been forced to use instruments that 
are partially inappropriate because of the limited choice available. 
The development of effective instruments can provide the keys that 
can unlock whole areas of science education research. 

We have learned again during this period of science curriculum 
development that the teacher is of central importance in a learning 
situation, probably more important than the materials, facilities, 
buildings or type of school organization. It may be that the 
teacher and how he teaches is more important in science instruction 
than in many other areas of the curriculum because so much of what 
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we ^nt to convey to students takes the form of general approaches 
to the universe, processes in dealing with problems, and attitudes 
toward fact and fancy. It may be that the most effective way to 
convey these elements of science instruction is through a teacher 
who can provide a model for his students. We need developmental 
studies in teacher education which will help future teachers bo 
learn how to provide such models. 

Implications and suggestions for further rese arch. The edu- 
cational researcher has the responsibility to suggest the implications 
of his research. We have an admirable ethic In research that bhe re- 
searcher should not go beyond his data in reporting his results. 
However, this can be carried to a cautions, conservative extreme. 

The researcher also has the responsibility to utilize his data tc 
the fullest. It is irresponsible for a science education researcher 
to undertake an important study and then retreat from suggesting the 
implications of his findings. Instead, the researcher owes it to 
other researchers and practitioners in the field to suggest what he 
thinks his findings imply. 

The results of research should be clearly stated, the bases 
for the statement of implications carefully formulated, and the 
implications of research unambiguously labelled. Other researchers 
and practitioners should be able to go to the research report, 
locate and understand the report of findings, check the author’s 
statement of implications against the findings, and possibly make 
their own interpretations. 

One of the most important results of research are the new 
questions that are raised and the new opportunities for research 
that are made evident. Science has been likened to an expanding 
sphere. As the sphere grows, the area for active and productive 
research increases. In science education research, the statement 
of "Suggestions for Further Research" should be taken seriously 
and regarded as one of the most important results of the research. 

The researcher can also suggest to practitioners ways that 
they can check his results in their situation. There is always 
danger in extrapolating to the universe. But, if we suggest ways 
our results can be checked, this danger is minimized. Also this 
is a way of linking action research on the local level to more 
basic research. It may also be a way of encouraging a research 
approach to decision-making in science education. 



